Directional and x-ray view techniques for navigation using a mobile device

ABSTRACT

Techniques for displaying navigation information on a mobile device are provided that include a method that includes obtaining an indication of a position and an indication of a direction associated with the mobile device, using the indication of the position, the indication of the direction, information regarding identities of POIs within a geographic region of interest, and information regarding areas associated with the POIs to determine at least one relevant POI, of the POIs, that is associated with the position and direction, and displaying at least one visual indication associated with each of the at least one relevant POI on the mobile device. The appearance of the at least one visual indication is dependent on at least one of a distance from the mobile device of the relevant POI associated with the visual indication or presence of a known physical barrier between the mobile device and that relevant POI.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a Continuation of U.S. application Ser. No.16/412,407, filed May 14, 2019, which is a Continuation of U.S.application Ser. No. 15/919,721, filed Mar. 13, 2018, which is aContinuation of U.S. application Ser. No. 13/777,861, filed Feb. 26,2013, all of which are hereby incorporated by reference, in theirentirety and for all purposes.

BACKGROUND

Indoor navigation presents issues different from outdoor navigation. Forexample, in complex indoor environments with large open spaces, such aslarge departments stores or sports stadiums, navigation may be differentthan in outdoor environments, e.g., because “streets” or “hallways” maynot be well defined, and a user may be permitted to move freely ornearly so. Providing directional guidance to a user in an indoorenvironment using outdoor environment techniques may therefore not workwell.

SUMMARY

An example method of displaying navigation information on a mobiledevice according to the disclosure includes obtaining, at the mobiledevice, an indication of a position of the mobile device, obtaining, atthe mobile device, an indication of a direction associated with themobile device, using the indication of the position of the mobiledevice, the indication of the direction associated with the mobiledevice, information regarding identities of points of interest within ageographic region of interest, and information regarding areasassociated with the points of interest to determine at least onerelevant point of interest, of the points of interest, that isassociated with the position and direction of the mobile device, anddisplaying at least one visual indication associated with each of the atleast one relevant point of interest on the mobile device. An appearanceof the at least one visual indication for each of the at least onerelevant point of interest is dependent on at least one of a distancefrom the mobile device of the relevant point of interest associated withthe visual indication or presence of a known physical barrier betweenthe mobile device and that relevant point of interest associated withthe visual indication.

Implementations of such a method may include one or more of thefollowing features. The at least one visual indication associated witheach of the at least one relevant point of interest has at least one ofa size or a transparency of the visual indication dependent on thedistance from the mobile device of the relevant point of interest. Theusing further comprises determining a first relevant point of interestat a first different distance from the mobile device and a secondrelevant point of interest at a second different distance from themobile device that is greater than the first distance, and thedisplaying further comprises displaying first and second visualindications corresponding to the first and second relevant points ofinterest, respectively, with the first visual indication being at leastone of larger or less transparent than the second visual indication. Theat least one visual indication associated with each of the at least onerelevant point of interest has a transparency of the visual indicationdependent on the presence of the known physical barrier between themobile device and the relevant point of interest, and wherein therelevant point of interest is disposed beyond the known physical barrierrelative to the mobile device and the visual indication is transparent.The geographic area of interests comprises an indoor environment. Usingthe indication of the position of the mobile device, the indication ofthe direction associated with the mobile device, information regardingidentities of points of interest within a geographic region of interest,and information regarding areas associated with the points of interestto determine the at least one relevant point of interest, of the pointsof interest, that is associated with the position and direction of themobile device includes: identifying at least one eye ray associated withthe position of the mobile device and the indication of direction of themobile device; projecting the eye ray through a geometric representationof the indoor environment; and identifying at least one point ofinterest with which the eye ray intersects as the at least one relevantpoint of interest. Displaying the at least one visual indicationassociated with each of the at least one relevant point of interest onthe mobile device includes: sorting the at least one relevant point ofinterest based on a distance of each of the at least one relevant pointof interest from the position of the mobile device; rendering at leastone visual indication associated with each of the relevant points ofinterest; and displaying the at least one visual indication associatedwith each of the at least one relevant point of interest and arepresentation of a field of view of a camera of the mobile device. Theat least one visual indication associated with each of the at least onerelevant point of interest has at least one of a size or a transparencyof the visual indication dependent on the distance from the mobiledevice of the relevant point of interest. Displaying the at least onevisual indication associated with each of the at least one relevantpoint of interest on the mobile device includes: sorting the at leastone relevant point of interest based on a distance of each of the atleast one relevant point of interest from the position of the mobiledevice; determining an alpha channel value for each of the at least onerelevant point of interest based on a distance of the at least onerelevant point of interest from the position of the mobile device;generating composite image content based on a graphical indicationsassociated with each of the at least one relevant point of interest andthe alpha channel value associated with each of the at least onerelevant point of interest; and displaying the composite image contenton a display of the mobile device. The graphical indications associatedwith each of the at least one point of interest comprises an image ofthe relevant point of interest associated with the graphicalindications.

An example apparatus for displaying navigation information on a mobiledevice according to the disclosure includes: means for obtaining, at themobile device, an indication of a position of the mobile device; meansfor obtaining, at the mobile device, an indication of a directionassociated with the mobile device; means for using the indication of theposition of the mobile device, the indication of the directionassociated with the mobile device, information regarding identities ofpoints of interest within a geographic region of interest, andinformation regarding areas associated with the points of interest todetermine at least one relevant point of interest, of the points ofinterest, that is associated with the position and direction of themobile device; and means for displaying a visual indication for each ofthe at least one relevant point of interest on the mobile device. Anappearance of the visual indication for each of the at least onerelevant point of interest is dependent on at least one of a distancefrom the mobile device of the relevant point of interest associated withthe visual indication or presence of a known physical barrier betweenthe mobile device and that relevant point of interest associated withthe visual indication.

Implementations of such an apparatus may include one or more of thefollowing features. The at least one visual indication associated witheach of the at least one relevant point of interest has at least one ofa size or a transparency of the visual indication dependent on thedistance from the mobile device of the relevant point of interest. Themeans for using includes means for determining a first relevant point ofinterest at a first different distance from the mobile device and asecond relevant point of interest at a second different distance fromthe mobile device that is greater than the first distance, and the meansfor displaying further includes means for displaying first and secondvisual indications corresponding to the first and second relevant pointsof interest, respectively, with the first visual indication being atleast one of larger or less transparent than the second visualindication. The at least one visual indication associated with each ofthe at least one relevant point of interest has a transparency of thevisual indication dependent on the presence of the known physicalbarrier between the mobile device and the relevant point of interest,and the relevant point of interest is disposed beyond the known physicalbarrier relative to the mobile device and the visual indication istransparent. The geographic area of interests comprises an indoorenvironment. The means for using the indication of the position of themobile device, the indication of the direction associated with themobile device, information regarding identities of points of interestwithin a geographic region of interest, and information regarding areasassociated with the points of interest to determine the at least onerelevant point of interest, of the points of interest, that isassociated with the position and direction of the mobile deviceincludes: means for identifying at least one eye ray associated with theposition of the mobile device and the indication of direction of themobile device; means for projecting the eye ray through a geometricrepresentation of the indoor environment; and means for identifying atleast one point of interest with which the eye ray intersects as the atleast one relevant point of interest. The means for displaying the atleast one visual indication associated with each of the at least onerelevant point of interest on the mobile device includes: means forsorting the at least one relevant point of interest based on a distanceof each of the at least one relevant point of interest from the positionof the mobile device; means for rendering at least one visual indicationassociated with each of the relevant points of interest; and means fordisplaying the at least one visual indication associated with each ofthe at least one relevant point of interest and a representation of afield of view of a camera of the mobile device. The at least one visualindication associated with each of the at least one relevant point ofinterest has at least one of a size or a transparency of the visualindication dependent on the distance from the mobile device of therelevant point of interest. The means for displaying the at least onevisual indication associated with each of the at least one relevantpoint of interest on the mobile device includes: means for sorting theat least one relevant point of interest based on a distance of each ofthe at least one relevant point of interest from the position of themobile device; means for determining an alpha channel value for each ofthe at least one relevant point of interest based on a distance of theat least one relevant point of interest from the position of the mobiledevice; means for generating composite image content based on agraphical indications associated with each of the at least one relevantpoint of interest and the alpha channel value associated with each ofthe at least one relevant point of interest; and means for displayingthe composite image content on a display of the mobile device. Thegraphical indications associated with each of the at least one point ofinterest comprises an image of the relevant point of interest associatedwith the graphical indications.

An example non-transitory computer-readable medium according to thedisclosure, has stored thereon computer-readable instructions fordisplaying navigation information on a mobile device. The instructionsare configured to cause a computer to obtain, at the mobile device, anindication of a position of the mobile device; obtain, at the mobiledevice, an indication of a direction associated with the mobile device;use the indication of the position of the mobile device, the indicationof the direction associated with the mobile device, informationregarding identities of points of interest within a geographic region ofinterest, and information regarding areas associated with the points ofinterest to determine at least one relevant point of interest, of thepoints of interest, that is associated with the position and directionof the mobile device; and display at least one visual indicationassociated with each of the at least one relevant point of interest onthe mobile device. An appearance of the at least one visual indicationfor each of the at least one relevant point of interest is dependent onat least one of a distance from the mobile device of the relevant pointof interest associated with the visual indication or presence of a knownphysical barrier between the mobile device and that relevant point ofinterest associated with the visual indication.

Implementations of such a non-transitory computer-readable medium mayinclude one or more of the following features. The at least one visualindication associated with each of the at least one relevant point ofinterest has at least one of a size or a transparency of the visualindication dependent on the distance from the mobile device of therelevant point of interest. The instructions configured to cause thecomputer to use include instructions to cause the computer to determinea first relevant point of interest at a first different distance fromthe mobile device and a second relevant point of interest at a seconddifferent distance from the mobile device that is greater than the firstdistance, and the instructions to cause the computer to display includeinstructions to cause the computer to display first and second visualindications corresponding to the first and second relevant points ofinterest, respectively, with the first visual indication being at leastone of larger or less transparent than the second visual indication. Theat least one visual indication associated with each of the at least onerelevant point of interest has a transparency of the visual indicationdependent on the presence of the known physical barrier between themobile device and the relevant point of interest, and wherein therelevant point of interest is disposed beyond the known physical barrierrelative to the mobile device and the visual indication is transparent.The geographic area of interests comprises an indoor environment. Theinstructions to cause the computer to use the indication of the positionof the mobile device, the indication of the direction associated withthe mobile device, information regarding identities of points ofinterest within a geographic region of interest, and informationregarding areas associated with the points of interest to determine theat least one relevant point of interest, of the points of interest, thatis associated with the position and direction of the mobile deviceinclude instructions to cause the computer to: identify at least one eyeray associated with the position of the mobile device and the indicationof direction of the mobile device; project the eye ray through ageometric representation of the indoor environment; and identify atleast one point of interest with which the eye ray intersects as the atleast one relevant point of interest. The instructions to cause thecomputer to display the at least one visual indication associated witheach of the at least one relevant point of interest on the mobile deviceinclude instructions to cause the computer to: sort the at least onerelevant point of interest based on a distance of each of the at leastone relevant point of interest from the position of the mobile device;render at least one visual indication associated with each of therelevant points of interest; and display the at least one visualindication associated with each of the at least one relevant point ofinterest and a representation of a field of view of a camera of themobile device. The at least one visual indication associated with eachof the at least one relevant point of interest has at least one of asize or a transparency of the visual indication dependent on thedistance from the mobile device of the relevant point of interest. Theinstructions to cause the computer to display the at least one visualindication associated with each of the at least one relevant point ofinterest on the mobile device include instructions to cause the computerto: sort the at least one relevant point of interest based on a distanceof each of the at least one relevant point of interest from the positionof the mobile device; determine an alpha channel value for each of theat least one relevant point of interest based on a distance of the atleast one relevant point of interest from the position of the mobiledevice; generate composite image content based on a graphicalindications associated with each of the at least one relevant point ofinterest and the alpha channel value associated with each of the atleast one relevant point of interest; and display the composite imagecontent on a display of the mobile device. The graphical indicationsassociated with each of the at least one point of interest comprises animage of the relevant point of interest associated with the graphicalindications.

An example apparatus for displaying navigation information on a mobiledevice according to the disclosure includes a transceiver configured totransmit and receive data wirelessly; a memory configured to storeprocessor-executable program code; and a processor. The processor isconfigured to obtain, at the mobile device, an indication of a positionof the mobile device; obtain, at the mobile device, an indication of adirection associated with the mobile device; use the indication of theposition of the mobile device, the indication of the directionassociated with the mobile device, information regarding identities ofpoints of interest within a geographic region of interest, andinformation regarding areas associated with the points of interest todetermine at least one relevant point of interest, of the points ofinterest, that is associated with the position and direction of themobile device; and display at least one visual indication associatedwith each of the at least one relevant point of interest on the mobiledevice. An appearance of the at least one visual indication for each ofthe at least one relevant point of interest is dependent on at least oneof a distance from the mobile device of the relevant point of interestassociated with the visual indication or presence of a known physicalbarrier between the mobile device and that relevant point of interestassociated with the visual indication.

Implementations of such an apparatus may include one or more of thefollowing features. The at least one visual indication associated witheach of the at least one relevant point of interest has at least one ofa size or a transparency of the visual indication dependent on thedistance from the mobile device of the relevant point of interest. Theprocessor being configured to use the indication of the position of themobile device, the indication of the direction associated with themobile device, information regarding identities of points of interestwithin a geographic region of interest, and information regarding areasassociated with the points of interest to determine at least onerelevant point of interest, of the points of interest, that isassociated with the position and direction of the mobile device isfurther configured to determine a first relevant point of interest at afirst different distance from the mobile device and a second relevantpoint of interest at a second different distance from the mobile devicethat is greater than the first distance, and wherein the processor beingconfigured to display at least one visual indication associated witheach of the at least one relevant point of interest on the mobile deviceis further configured to display first and second visual indicationscorresponding to the first and second relevant points of interest,respectively, with the first visual indication being at least one oflarger or less transparent than the second visual indication. The atleast one visual indication associated with each of the at least onerelevant point of interest has a transparency of the visual indicationdependent on the presence of the known physical barrier between themobile device and the relevant point of interest, and wherein therelevant point of interest is disposed beyond the known physical barrierrelative to the mobile device and the visual indication is transparent.The geographic area of interests comprises an indoor environment. Theprocessor being configure to use the indication of the position of themobile device, the indication of the direction associated with themobile device, information regarding identities of points of interestwithin a geographic region of interest, and information regarding areasassociated with the points of interest to determine the at least onerelevant point of interest, of the points of interest, that isassociated with the position and direction of the mobile device isfurther configured to: identify at least one eye ray associated with theposition of the mobile device and the indication of direction of themobile device; project the eye ray through a geometric representation ofthe indoor environment; and identify at least one point of interest withwhich the eye ray intersects as the at least one relevant point ofinterest. The processor being configured to display at least one visualindication associated with each of the at least one relevant point ofinterest on the mobile device is further configured to: sort the atleast one relevant point of interest based on a distance of each of theat least one relevant point of interest from the position of the mobiledevice; render at least one visual indication associated with each ofthe relevant points of interest; and display the at least one visualindication associated with each of the at least one relevant point ofinterest and a representation of a field of view of a camera of themobile device. The at least one visual indication associated with eachof the at least one relevant point of interest has at least one of asize or a transparency of the visual indication dependent on thedistance from the mobile device of the relevant point of interest. Theprocessor being configured to display the at least one visual indicationassociated with each of the at least one relevant point of interest onthe mobile device is further configured to: sort the at least onerelevant point of interest based on a distance of each of the at leastone relevant point of interest from the position of the mobile device;determine an alpha channel value for each of the at least one relevantpoint of interest based on a distance of the at least one relevant pointof interest from the position of the mobile device; generate compositeimage content based on a graphical indications associated with each ofthe at least one relevant point of interest and the alpha channel valueassociated with each of the at least one relevant point of interest; anddisplay the composite image content on a display of the mobile device.The graphical indications associated with each of the at least one pointof interest comprises an image of the relevant point of interestassociated with the graphical indications.

Items and/or techniques described herein may provide one or more of thefollowing capabilities, as well as other capabilities not mentioned.

BRIEF DESCRIPTIONS OF THE DRAWINGS

FIG. 1 is a simplified diagram of a wireless communication system.

FIG. 2 is a functional block diagram of a mobile device that can be usedto implement the mobile devices illustrated in FIG. 1.

FIG. 3 is a functional block diagram of the mobile device illustrated inFIG. 1 that illustrates functional modules of a memory shown in FIG. 2.

FIG. 4 is a functional block diagram of a computer system that can beused to implement the navigation server illustrated in FIG. 1.

FIG. 5 is a functional block diagram of a computer system that can beused to implement the location server illustrated in FIG. 1.

FIG. 6 is a block flow diagram of a process for displaying navigationinformation on a mobile device.

FIG. 7 is a block flow diagram of a process for determining a relevantset of points of interest.

FIG. 8 is a block flow diagram of a process for determining displayingon the display of the mobile device a directional view of relevantpoints of interest within a field of view of a camera of a mobiledevice.

FIG. 9 is a block flow diagram of a process for determining displayingan X-ray view of points of interest within a field of view of a cameraof a mobile device on a display of the mobile device.

FIG. 10 provides an example of a mobile device displaying a directionalview of POIs within the field of view of the camera of the mobiledevice.

FIGS. 11A, 11B, 11C, and 11D provide an example of a how an X-ray reviewof a geographical area of interest can be created from an image of thearea of geographical interest (FIG. 11A), an image associated with a POI(FIG. 11B), an illustration of an eye-ray being projected from thecurrent position of the mobile device through a 2D model of thegeographic area of interest (FIG. 11C), and an X-ray view generatedbased on the image of the geographical area of interest and the imageassociated with the POI (FIG. 11D).

DETAILED DESCRIPTION

Techniques are provided for assisting with navigation through an indoorenvironment and/or through a combination of indoor and outdoorenvironments using a mobile device. The indoor and/or combination ofindoor and outdoor environment for which the navigation techniqueprovided herein are generally referred to as a “geographical area ofinterest.” The techniques provided can assist with navigation through ageographical area of interest comprising an indoor environment, such asan office building, shopping mall, a hotel, government offices, eventvenue, or other indoor environment. The techniques provided can alsoassist with navigation thorough a geographical area of interestcomprising a combination of indoor and outdoor environments, such as acorporate campus or a university campus that are a combination of bothindoor and outdoor spaces. The techniques provided could also be appliedto other areas where a user may wish to navigate from an outdoor area toan indoor area or vice versa, such as downtown area or a shoppingdistrict where the user may wish to navigate from a street level intoone or more buildings or vice versa and to be able to view POIs withinthe geographical area of interest.

The navigation techniques provided can present visual indications ofpoints of interest (POIs) to a user of a mobile device. The POIspresented to the user can include POIs that are not visible to the userand/or to a camera of the mobile device. The visual indications of thePOIs can include textual and/or graphical indications of the POIs. Thevisual indications of the POIs can be provided in the direction of thePOIs from the mobile device and can include indications of how far thePOIs are from the position of the mobile device. The visual indicationscan also indicate whether each POI is beyond a physical barrier (e.g., awall, floor, or ceiling) of a building within the geographic area ofinterest. The visual indications can provide a sense of distances of thePOIs from the position of the mobile device, e.g., with decreasing sizeof indications with increasing distance from the mobile device and/orincreasing transparency of the indications with increasing distance fromthe mobile device.

In an example implementation, a mobile device, such as a mobile phone ortablet computer, can determine the position and orientation of themobile device. Using the position and orientation, the mobile device canbe configured determine a viewing direction or field of view (FOV) of acamera of the mobile device. The mobile device can compare the FOV witha two-dimensional (2D) or three-dimensional (3D) map that includeslocations and areas corresponding to the POIs. The mobile device canprovide visual indications on a display of the mobile device as to thePOIs that are within the FOV of the camera of the mobile device based onthe current position and orientation of the mobile device.

FIG. 1 is a simplified diagram of an example wireless communicationsystem in which the techniques for improved navigation of an indoorenvironment disclosed herein can be implemented. In the exampleillustrated in FIG. 1, the wireless network environment includeswireless access points 115 (where each instance of a wireless accesspoint is labeled with a separate letter—e.g., 115 a, 115 b, and 115 c),a base station 140, a plurality of mobile devices 120 (where eachinstance of which is labeled with a separate letter—e.g., 120 a, 120 b),a network 110, a navigation server 125, a location server 150.

The example network environment illustrated in FIG. 1 includes a threewireless access points 115 a, 115 b, and 115 c, but other networkenvironments that can be used to implement the techniques disclosedherein may include more wireless access points. Furthermore, thewireless access points may be distributed throughout an indoorenvironment and may be placed on different floors within the indoorenvironment. The term “access point” is used for simplicity, but refersto communication devices, more generally, one example of which includeaccess points in wireless local area networks, such as IEEE 802 seriescompliant networks including the IEEE 802.11 family of standardscommonly referred to as Wi-Fi. For example, devices that use Bluetoothcan be communication devices according to the disclosure. As such, theusage of the term access point is not exclusive of other kinds ofcommunication networks. Furthermore, a wireless access point 115 can beconfigured to implement other wireless communications standards inaddition or instead of standards from the IEEE 802 series. The wirelessaccess point 115 can also be implemented as a femtocell or picocell thatis configured to operate a base station for a mobile communicationsnetwork. A wireless access point 115 can either be located at a fixedlocation or may be mobile. For example, a wireless access point 115 canbe a mobile device that is configured to provide a WiFi hotspot and isconfigured to wirelessly connect to a mobile communication network, e.g.a WAN to provide network connectivity to the WiFi hotspot.

The base station 140 can be configured to wirelessly communication witha plurality of mobile devices, such as mobile devices 120 a and 120 b.The base station 140 may be located within the indoor environment and/ormay provide a coverage area that overlaps with at least a portion of theindoor environment. The base station 140 can be associated with a mobilecommunications network provider and can be configured to communicationusing one or more wireless communications protocols for mobile devices.For example, the base station 140 can be configured to use on more ofthe wireless communications protocols, such as Code Division MultipleAccess (CDMA), Time Division Multiple Access (TDMA), Frequency DivisionMultiple Access (FDMA), Orthogonal FDMA (OFDMA), and Single-Carrier FDMA(SC-FDMA), Long Term Evolution (LTE), and/or other wirelesscommunications protocols. For example, the base station 140 can be anevolved Node B (eNB) in a wireless communication system implanting theLTE wireless communications standards. The wireless communicationsdiscussed herein are only examples of some of the types of wirelessprotocols that may be implemented by the base station 140, and are notintended to limit the techniques discussed herein to wirelesscommunication networks using these protocols. The network environmentillustrated in FIG. 1 only illustrates a single base station 140, but anindoor environment such as those where the techniques disclosed hereinmay be implemented may be within the coverage area of multiple basestations 140.

The example network environment illustrated in FIG. 1 also includes twomobile devices 120 a and 120 b, but other network environments that canbe used to implement the techniques disclosed herein may include eithermore or less mobile devices. The mobile devices 120 a and 120 b can beconfigured to communicate with one or more servers, such as navigationserver 125 and/or location server 150. The mobile devices 120 a and 120b can be configured to communicate with the servers through one of thewireless access points 115 or the wireless base station 140.

The navigation server 125 can be configured to store electronic content,such as applications, that can be provided to one or more mobile devices120 and can be executed by the mobile devices 120. In otherimplementations, the network environment may contain other servers (notshown) that can also be configured to provide information and/orelectronic content to the mobile devices 120 a and 120 b. The navigationserver 125 can be configured to provide 2D and/or 3D map information forindoor environments to the mobile devices 120. The navigation server 125may be implemented by an owner or manager of an indoor environment ormay be implemented by a third party, such as a wireless network serviceprovider. An example implementation of a navigation server that can beused to implement the navigation server 125 is provided in FIG. 4.

The location server 150 can be configured to provide location basedservices, such as determining the position of the mobile device 120and/or providing assistance data that the mobile devices 120 can use todetermine the position of the mobile devices 120. An exampleimplementation of a navigation server that can be used to implement thenavigation server 125 is provided in FIG. 5. In some implementations,the navigation server 125 and the location server 150 can be implementedon the same server.

FIG. 2 is an example of a system architecture that can be used toimplement a mobile device, such as the mobile devices 120 a and 120 billustrated in FIG. 1. A mobile device 120 may include some or all ofthe components illustrated in FIG. 2 and may include additionalcomponents illustrated in FIG. 3 and can be configured to implementvarious functions, such as the processes illustrated in FIGS. 6-9described in detail below.

The mobile device 120 can include sensors 205 (e.g., gyros,accelerometers, an inertial measurement unit (IMU) and/or other types ofsensors) and a sensor processor 210 for processing data collected by thesensors 205. Data collected by the sensors or the camera may be used todetermine an orientation of the mobile device, acceleration or velocityof the mobile device, and/or other information that may be useful indetermining the location of the mobile device 120. The data collected bythe sensor 205 can also be used to provide inputs to one or moreapplications on the mobile device 120. The mobile device 120 can beconfigured to use the data collected by the sensors 205 to determine howthe mobile device 120 is oriented and can use that information inaddition to the estimated position of the mobile device within an indoorenvironment to determine a field of view associated with the mobiledevice within the indoor environment and to display that field of viewon the mobile device 120. The field of view into the indoor environmentcan provide the user with an augmented view of the indoor environment inwhich they are located that can assist the user in navigating throughthe indoor environment.

The mobile device 120 can also include a camera 215 and a cameraprocessor 220. The camera 215 can be configured to capture images and/orvideo content. The camera processor 220 can be configured to process thedata collected by the camera 215 and to convert the data collected bythe camera into a format that can be used by one or more applications onthe mobile device 120 and/or viewed on the display 225 of the mobiledevice 120. The camera processor 220 can be configured to performvarious types of image or video processing on the data collected fromthe camera to prepare the content for display on display 225. Forexample, the camera of the mobile device 120 can be configured tocapture images and/or video within the field of view of the camera 215of the mobile device 120. The mobile device 120 can be configured to usethe captured images and/or video to provide an augmented view of theindoor environment to the user of the mobile device that can includepoint of interest (POI) information. For example, the mobile device 120can be configured to provide a directional view that can include textualand/or graphical indications identifying the position of POIs relativeto the position of the mobile device 120 and/or an X-ray view where POIinformation for POIs that may otherwise be occluded from view bycomponents of the indoor environment, such as doors, walls, ceilings,and floors, can be displayed to the user. Camera processor 220 may alsobe used to determine how the mobile device 120 is oriented.

The display 225 can be a touch screen interface that includes touchsensor 235. The graphics processor 230 can be used to generate graphicaldata for display on display 225. Applications on the mobiles station 12and/or other executable programs, such as the operating system and thepositioning engine, can be implemented in software and/or hardware andcan be configured to send commands to the graphics processor 230 todisplay image or video content on the display 225. The touch sensorprocessor 240 can process data output by the touch sensor 235 toidentify when a user touches the touch screen. The touch sensorprocessor 240 can be configured to identify various touch gestures,including multi-finger touches of the touch screen. The operatingsystem, applications, and/or the positioning engine can use the gestureinformation determined by the touch sensor processor 240 to determine,at least in part, receive and respond to user input. The display 225 ofthe mobile device 120 can be a 2D display or can be a 3D display capableof displaying stereoscopic images. The mobile device 120 can beconfigured to render 2D and/or 3D content depending upon thecapabilities of the display 225 and the configuration of the mobiledevice 120.

The communications controller 250 can be configured to enable the deviceto communicate using one or more wireless protocols and to enable themobile device to communicate with one more devices via the network 110.The communications controller 250 can be configured to allow the deviceto send and receive data from nearby wireless devices, includingwireless access point and other mobile devices 120. For example, thecommunications controller 250 can be configured to enable the mobiledevice 120 to communicate either directly or indirectly with thewireless access points 115, the base station 140, the navigation server125, the location server 150, and other mobile devices 120. Thecommunications controller 250 can also be configured to enable themobile device 120 to receive navigation signals that the mobile device120 can use to determine its location. For example, the communicationscontroller 250 can be configured to receive signals from satellitevehicles (SVs) belonging to one or more Satellite Positioning Systems(SPSs), such as the GPS system, the GLONASS system, the Galileo system,and/or other SPSs.

The memory 260 includes volatile and/or persistent memory for storingdata used by various components of the mobile device 120. The memory 260can be used to store processor-executable program code for one or moreof the processors included in the device. The processor controller 255can be configured to control one or more of the sensor processor 210,camera processor 220, the graphics processor 230, and the touch sensorprocessor 240. One or more of the sensor processor 210, camera processor220, the graphics processor 230, and the touch sensor processor 240 mayalso be implemented by the processor controller 255. The processorcontroller 255 can also be configured to execute processor-executableprogram code.

FIG. 3 is a functional block diagram of the mobile device illustrated inFIG. 1 that illustrates functional modules of a memory shown in FIG. 2.For example, the mobile device 120 can include a rendering module 362, apoint of interest module 364, an orientation determination module 366, adata access module 368, and a position determination module 370. Themobile device 120 can also include a navigation database 350.

The rendering module 362 can be configured to render an augmented viewof an indoor environment that can improve navigation through ageographic area of interest. The rendering module 362 can configured torender a view of an indoor and/or outdoor environment that overlaysand/or blends augmented content with images and/or video captured by thecamera 215 of the mobile device 120. The rendering module 362 can beconfigured to display a representation of the “field of view” of thecamera 215 of the mobile device 120 on the display 225 of the mobiledevice 120. The camera can be configured to capture a series of stillimages or video of a portion the geographic area of interest within thefield of view of the camera 215 and the rendering module 362 can beconfigured to receive the series of still images and/or video capturedby the camera and to augment the received content with navigationalaids. The navigational aids can provide visual indications of thelocations of points of interest (POIs) relative to the position of themobile device 120 within the indoor environment. The rendering module362 can be configured to provide a directional view of the geographicalarea of interest and an X-ray view of the geographical area of interestthat can provide navigational clues to a user of the mobile device 120that can assist the user of the mobile device 120 in navigating theindoor and/or environments comprising the geographic area of interest.The mobile device 120 can be configured to display the directional viewand/or the X-ray view.

The rendering module 362 can be configured to obtain the currentposition of the mobile device 120 from the position determination module370 and/or the location server 150 and the orientation of the mobiledevice 120 from the orientation determination module 366. The renderingmodule 362 can be configured to provide the position and orientation ofthe mobile device to the point of interest (POI) module 364 to obtainPOI information for points of interest within the field of view of themobile device 120. The POI information can be sorted by relativedistance of each of the POIs from the current position of the mobiledevice 120 and can include a graphical and/or textual indicationsassociated with each of the POIs.

When the mobile device 120 is configured to operate in the directionalview, the rendering module 362 can be configured to provide an augmentedview of the geographic area of interest that includes overlays oftextual and/or graphical indications illustrating the relative positionof POIs within the geographic area of interest. Once the renderingmodule 362 has obtained the POIs from the POI module 364, the renderingmodule 362 can be configured to generate the augmented view of thegeographic area of interest by overlaying the graphical and/or textualindications associated with each POI within the field of view of thecamera onto the series of images and/or video content captured by thecamera 215 of the mobile device 120. For example, when used in an indoorshopping mall, the directional view of an indoor environment could beused to display POIs typically founds in a shopping mall, such asrestrooms, fitting rooms, store locations, and mall exits. FIG. 10provides an example of a mobile device 120 displaying a directional viewof POIs within the field of view of the camera 215 of the mobile device120.

The rendering module 362 can be configured to include the textual and/orgraphical indications associated with each of the POIs in the field viewin the augmented view of the indoor environment generated by therendering module 362. The rendering module 362 can be configured toplace the textual and/or graphical indications associated with each ofthe POIs within the augmented view of the indoor environment based ontheir position of each of the POIs relative to the current position ofthe mobile device 120. The rendering module 362 can be configured torender the textual and/or graphical indications associated with POI toindicate the relative distance of the POI from the current position ofthe mobile device 120. For example, the rendering module 362 can beconfigured render the textual and/or graphical indications associatedwith POIs that are closer to current position of the mobile device 120such that the textual and/or graphical indications are larger than thetextual and/or graphical indications associated with POIs that arefarther from the current position of the mobile device 120. Therendering module 362 can also be configured to use other methods toindicate the distance of the POIs from the current position of themobile device, such as color coding the textual and/or graphicalindications, rendering the textual and/or graphical indications usingdifferent levels of transparency based on the distance of the POIs fromthe current position of mobile device, or using different textures whenrendering the textual and/or graphical indications to representdifferent distances from the current position of the mobile device 120.For example, the textual and/or graphical indications associated withPOIs that are closer to the current position of the mobile device 120can be rendered as opaque while the textual and/or graphical indicationsassociated with POIs that are farther from the mobile device 120 can berendered as increasingly transparent based on their distance from thecurrent position of the mobile device 120.

When the mobile device 120 is configured to operate in the X-ray view,the rendering module 362 can be configured to provide an augmented viewof the indoor environment that can illustrate details of POIs that wouldotherwise be blocked by objects in the field of view of the mobiledevice 120, including components of the indoor environment and/or otherPOIs. When rendering the X-ray view, the rendering module 362 can beconfigured to first obtain a set of POIs from the POI module 364 asdescribed above. Each POI can include one or more images associated withthe POI, and the rendering module 362 can be configured to render acomposite image of the indoor environment based on the series of imagesand/or video captured by the mobile device 120 and the images associatedwith the POIs that fall within the field of view of the mobile device120. The rendering module 362 can be configured to request that the POIsretrieved by the POI module 364 be ordered based on the distance betweeneach of the POIs and the current position of the mobile device 120. Therendering module 362 can then generate an augmented view where an imageassociated with the POI is rendered as semi-transparent based on thedistance of the POI from the current position of the mobile device 120.In one implementation, the rendering module 362 can be configured to usealpha compositing to generate the augmented view where the imageassociated with each of the POIs is associated with an alpha channelvalue that falls within a range of completely opaque to completelytransparent. The rendering module 362 can be configured to assign analpha channel value representing a higher transparency to those POIsfarther from the current position of the mobile device 120 and an alphachannel value representing a lower transparency to those POIs closer tothe current position of the mobile device 120. The rendering module 362can then be configured to render a composite image of the field of viewof the mobile device 120 by compositing the field of view contentcaptured by the camera 225 with the POI images, starting with those POIsfarther from the current position of the mobile device 120. Theresulting composite image provides X-ray view of the indoor environmentthat can allow the user to see through walls of the indoor environmentto see details that the user would not otherwise be able to see fromtheir vantage point.

FIGS. 11A, 11B, 11C, and 11D provide an example of a how an X-ray reviewof a geographical area of interest can be created from an image of thearea of geographical interest. The image illustrated in FIG. 11A is animage of a geographical area of interest captured by the camera 215 ofthe mobile device 120. In this example, the geographical area ofinterest is a shopping mall, and the image illustrated in FIG. 11A is ofa corridor of the shopping mall. FIG. 11B is an image associated withthat a POI. In this example, the POI is an image of the interior of adepartment store within the shopping mall. FIG. 11C is a figureillustrating projecting an eye ray from the current position of themobile device 120 through a 2D map of the geographic area of interest.The 2D map in this example is a 2D map of the shopping mall. The eye rayprojection technique is illustrated in FIG. 7 and is discussed ingreater detail below. The process illustrated in FIG. 7 can also be usedto identify POIs relevant POIs that can be displayed using thedirectional view. FIG. 11D provides an example of an X-ray view of aPOI. In this example, the view of the corridor representing the field ofview of the camera 215 of the mobile device 120 that is illustrated inFIG. 11A has been rendered as transparent. The relevant POIs can then beoverlaid over the image of the field of view of the camera 215 andrendered as partially transparent. In the example illustrated in FIG.11D, only one POI has been included. However, if multiple relevant POIswere identified within the field of view of the camera 215, the POIscould be sorted based on their distance from the current position of themobile device 120, and the textual and/or graphical indicationsassociated with those POIs can then be rendered from farthest away toclosed to the current position of the mobile device 120. A decreasingtransparency level (alpha channel value) can be applied to the POIs thatare closer to the current position of the mobile device 120. The X-rayview illustrated in FIG. 11D provides the user of the mobile device 120with a view through the walls of the corridor to see a representation ofthe department store that is behind the walls of the corridor.

The rendering module 362 can be configured to request and receiveupdates to the position of the mobile device 120 from the positiondetermination module 370 and/or the location server 150. For example,the rendering module 362 can be configured to automatically receiveupdates from the position determination module 370 and/or the locationserver 150 when the position of the mobile device 120 changes. Therendering module 362 can also be configured to receive updates from theorientation determination module 366 when the orientation of the mobiledevice changes 120. The rendering module 362 can be configured toautomatically update the augmented view (directional view, X-ray view,or other augmented view) generated for display on the display 225 of themobile device 120 in response to changes to the position and/or theorientation of the mobile device 120.

The point of interest (POI) module 364 can be configured to identifypoints of interest within the field of view of the mobile device 120.The rendering module 362 can be configured to obtain the currentposition of the mobile device 120 within the geographical area ofinterest from the position determination module 370 and/or from thelocation server 150 and to provide the location of the mobile device 120to the point of interest module 364. The rendering module 362 can alsobe configured to obtain orientation information for the mobile device120 from the orientation determination module 366. The POI module 364can be configured use the position and orientation information toidentify a set of one or more POIs that are in the direction of thefield of view of the mobile device 120. The POIs may or may not bevisible from the current position of the mobile device 120. The POIs maybe behind other objects in the field of view of the camera. For example,the POIs may be behind physical barriers, such as components of thegeographical area of interest, including walls, doors, floors, ceilings,or other components of the geographical area of interest. The POI module364 can be configured to order the set of POIs by their relativedistance from the current position of the mobile device 120 and can beconfigured to mark each POI to indicate whether the POI may be visiblefrom the current position of the mobile device or whether the POI isobscured behind another component of the geographical area of interest.

The POI module 364 can be configured to access POI data from thenavigation database 350 via the data access module 368. The data accessmodule 368 can be configured to access the POI information for thegeographical area of interest, and to provide the POI data to the POImodule 364. For example, the data access module 368 can be configured toaccess the POI information for a particular indoor environment orportion of an indoor environment associated with the geographic area ofinterest, such as a floor of the indoor environment.

The navigation database 350 can include POI information associated withPOIs. For example, the POI information associated with the POI caninclude geometry information defining the boundaries of the POI. Forexample, the POI information can include a set of one or more polygonsdefining the boundaries of the POI. The POI information can also includecoordinates associated with the POI. The coordinates can be expressed ingeographic coordinates (e.g., latitude and longitude) or expressed in aself-defined coordinate system where the coordinates of each of the POIsfor the geographic area of interest is normalized to a bounding area.The information associated with the POI can also include images of thePOIs that can be used to represent the POI in the X-ray view of thegeographic area of interest and can include textual and/or graphicallabels that can be used to represent the POI in the directional view ofthe geographic area of interest provided by the rendering engine 362.

The POI module 364 can be configured to identify which POIS are relevantusing a process similar to that illustrated in FIG. 7. The POI module364 can be configured to project an “eye ray” through a 2D or 3Dgeometric representation of the geographic area of interest. The eye raycan be projected through the geometric representation of the geographicarea of interest and those POIs which with the eye ray intersect can beselected as relevant POIs. The relevant POIs may not directly be withinthe line of sight of the camera 215 of the mobile device 120 and may beseparated from the position of the mobile device 120 by one morephysical barriers, including components of the geographic area ofinterest, such as walls, floors, ceiling, doors, etc. or by one moreobjects within the geographic area of interest.

The orientation determination module 366 can be configured to receiveinformation collected by the sensors 205. The data collected by thesensors 205 can be received and processed by the sensor processor 210and/or may be transferred to the orientation determination module 366 bythe sensor processor 210 as the sensor data is received from the sensors205. The orientation determination module 366 can be configured toprovide sensor data to the rendering module 362, the positiondetermination module 370, the field of view module 372, and/or othermodules or applications on the mobile device 120. For example, theorientation determination module 366 can be configured to determine anorientation of the mobile device 120 based on information collected bythe sensors 205 and/or the camera 215, and the orientation determinationmodule 366 can provide the orientation to the rendering module 362and/or the field of view module 372. In some implementations, theorientation determination module 366 can be configured to determine theorientation of the mobile device based on image content captured by thecamera 215 of the mobile device 120. The orientation determinationmodule 366 can be configured to use various techniques, such as thedetermining a vanishing point for an image captured by the camera 215 ofthe mobile device 120 to determine the orientation of the mobile device120.

The field of view module 372 can be configured to use the orientationinformation to determine a field of view of the mobile device 120, andthe field of view module 372 can provide the field of view informationto the point of interest module 364 which can be configured to use thefield of view information to determine which POIs are included in thefield of view of the mobile device 120. The rendering module 362 can beconfigured to use the field of view information 372 to determine whichportion of the indoor environment should be rendered on the display 225of the mobile device 120.

The data access module 368 can be configured to access data stored inthe memory 460 of the mobile device 120 and/or in the navigationdatabase 350. The data access module 368 can be configured to accessdata stored in or to store data in the memory 260 and/or in thenavigation database 350 in response to requests from the renderingmodule 362, the orientation determination module 366, the field of viewmodule 372, and/or other applications or modules implemented on themobile device 120.

The position determination module 370 can be configured to determine theposition of the mobile device 120 within an indoor environment and/or inan outdoor environment. In an indoor environment, the positiondetermination module 370 can be configured to obtain measurements ofsignals from a plurality of wireless access points 115 to estimate thelocation of the mobile device 120 within an indoor environment invarious ways. The position determination module 370 can be configured todetermine the position of the mobile device 120 using signals fromwireless access points 115 to determine the position of the mobiledevice 120 within an indoor environment. For example, the positiondetermination module 370 can be configured to perform trilaterationusing signal measurements (e.g., RSSI (received signal strengthindication), RTT (round-trip time)), time of arrival (TOA), and/ordisambiguate between multiple possible disambiguation regions (e.g.,floors) within an indoor environment, e.g., using RSSI and/or RTTmeasurements and known locations of the wireless access points 115and/or wireless base stations 140. The position determination module 370can also be configured to determine the position of the mobile device120 using signals received from a Satellite Positioning System receiverassociated with the mobile device 120 (e.g., associated withcommunications controller 250) to determine the position of the mobiledevice 120 when outdoors and a line of site to a sufficient number ofsatellites from one or more SPS constellations.

The position determination module 370 can also be configured to provideinformation to the location server 150 that the location server 150 canuse to determine the location of the mobile device 120 and/or that thelocation server 150 can use to provide assistance data to the mobiledevice 120 such that the mobile device 120 determine the location of themobile device 120. For example, location server 150 can send assistancedata to the mobile device 120 that the mobile device 120 can use toacquire signals from satellites from one or more SPS constellations. Thelocation server 150 can also be configured to provide the positiondetermination module 370 of the mobile device 120 with an almanac ofwireless access points 115 and/or base stations 140 which may beproximate to the mobile device 120. The position determination module370 can use this information to identify wireless access points 115and/or base stations 140 that may be proximate to the mobile device 120and to use signals from nearby wireless access points 115 and/or basestations 140 to determine the position of the mobile device 120.

The position determination module 370 can be configured to provide theposition of the mobile device 120 within an indoor environment to therendering module 362, which can use the position information and thefield of view information obtained from the field of view module 372 todetermine what to render in the augmented view of the indoor environmentpresented on the display 225 of the mobile device 120.

FIG. 4 is a functional block diagram of a computer system that can beused to implement the navigation server 125 illustrated in FIG. 1. Thenavigation server 125 can include a processor 455, a network interface450, a memory 460, and a navigation database 450. The navigation server125 can also include functional modules that can be implemented asprocessor executable software code stored in memory 460. The functionalmodules illustrated in FIG. 4 can alternatively be stored either in partor in whole in a different computer-readable memory. For example, thefunctional modules may be stored in a removable computer-readable mediumassociated with the navigation server 125 120, such as a flash memory, aSD-Card, or other type of removable memory. The functional modulesillustrated in FIG. 4 can also be implemented in part or in whole infirmware and/or hardware. The navigation server 125 can include arendering module 462, a point of interest module 464, and a data accessmodule 466.

The processor 455 can be configured to execute software code stored inthe memory 460. For example, the processor 455 can be configured toexecute the software code associated with the rendering module 462, thepoint of interest module 464, and the data access module 466. Theprocessor 455 can also be configured to execute other software code usedby the navigation server 125, such as operating system components,software drivers for peripheral devices, and/or other software code thatmay be executed by the navigation server 125 but the description ofwhich is not required to understand the operation of the systems andmethods disclosed herein.

The network interface 450 can be configured to provide bidirectionalwireless and/or wired network communications to the navigation server125. The network interface 450 can be configured to allow the navigationserver 125 to communicate with mobile devices 120 via one or moreintermediate networks, such as the Internet, a wireless network serviceprovider's core network, one or more wireless local area networks(WLANs), and/or other types of network. The network communicationsbetween the network interface 450 of the navigation server 125 and themobile devices 120 may be routed through one or more other networkelements, such as wireless access point 115 or base station 140.

The memory 460 can comprise volatile and/or persistent memory forstoring data used by various components of the navigation server 125.The memory 460 can be used to store processor-executable program codefor one or more of the processors included in the navigation server 125,such as processor 455.

The rendering module 462 can be configured to operate similarly to therendering module 362 of the mobile device 120. The rendering module 462can be configured to render an augmented view of an indoor environmentthat can improve navigation through the indoor environment. In someimplementations, the navigation server 125 can be configured to renderand provide the augmented navigation content to the mobile device 120rather than the augmented navigation content being generated on themobile device 120. The augmented navigation content may be performed onthe navigation server 125 rather than the mobile device 120 for variousreasons. For example, the augmented navigation content may be generatedon the navigation server 125 the mobile device 120 may not havesufficient resources to render the augmented navigation content or theaugmented navigation content may be rendered by the navigation server125 to conserve limited processing power, memory, and/or battery life onthe mobile device 120.

The rendering module 462 can be configured to render a view of theindoor environment that overlays and/or blends augmented content withimages and/or video captured by the camera 215 of the mobile device 120.The rendering module 462 can be configured to render a representation ofthe “field of view” of the camera 215 of the mobile device on thedisplay 225 of the mobile device. The camera can be configured tocapture a series of still images or video of a portion the indoorenvironment within the field of view of the camera 215 and the renderingmodule 462 can be configured to receive the series of still imagesand/or video captured by the camera and to augment the received contentwith navigational aids. The navigational aids can provide visualindications of the locations of points of interest (POIs) relative tothe position of the mobile device 120 within the indoor environment. Therendering module 462 can be configured to provide a directional view ofthe indoor environment and an X-ray view of the indoor environment thatcan provide navigational clues to a user of the mobile device 120 thatcan assist the user of the mobile device in navigating the indoorenvironment.

The rendering module 462 can be configured to obtain the currentposition of the mobile device 120 from the position determination module370 and/or the location server 150 and the orientation of the mobiledevice 120 from the orientation determination module 366. The renderingmodule c can be configured to provide the position and orientation ofthe mobile device to the point of interest (POI) module 364 to obtainPOI information for points of interest within the field of view of themobile device 120. The POI information can be sorted by relativedistance of each POI from the current position of the mobile device 120and can include a graphical and/or textual indications associated witheach POI.

When the mobile device 120 is configured to operate in the directionalview, the rendering module 462 can be configured to provide an augmentedview of the indoor environment that includes overlays of textual and/orgraphical indications illustrating the relative position of POIs withinthe indoor environment. Once the rendering module 362 has obtained thePOIs from the POI module 464, the rendering module 462 can be configuredto generate the augmented view of the indoor environment by overlayingthe graphical and/or textual indications associated with each POI withinthe field of view of the camera onto the services of images and/or videocontent captured by the camera 215 of the mobile device 120. Forexample, when used in an indoor shopping mall, the directional view ofan indoor environment could be used to display POIs typically founds ina shopping mall, such as restrooms, fitting rooms, store locations, andmall exits. FIG. 10, as described above, provides an example of adirectional view of a geographical area of interest, which in thisexample is a shopping mall.

The rendering module 462 can be configured to include the textual and/orgraphical indications associated with each of the POIs in the field viewin the augmented view of the indoor environment generated by therendering module 462. The rendering module 462 can be configured toplace the textual and/or graphical indications associated with each ofthe POIs within the augmented view of the indoor environment based ontheir position of each of the POIs relative to the current position ofthe mobile device 120. The rendering module 462 can be configured torender the textual and/or graphical indications associated with POI toindicate the relative distance of the POI from the current position ofthe mobile device 120. For example, the rendering module 462 can beconfigured render the textual and/or graphical indications associatedwith POIs that are closer to current position of the mobile device suchthat the textual and/or graphical indications are larger than thetextual and/or graphical indications associated with POIs that arefarther from the current position of the mobile device 120. Therendering module 362 can also be configured to use other methods toindicate the distance of the POIs from the current position of themobile device, such as color coding the textual and/or graphicalindications, rendering the textual and/or graphical indications usingdifferent levels of transparency based on the distance of the POIs fromthe current position of mobile device, or using different textures whenrendering the textual and/or graphical indications to representdifferent distances from the current position of the mobile device 120.For example, the textual and/or graphical indications associated withPOIs that are closer to the current position of the mobile device 120can be rendered as opaque while the textual and/or graphical indicationsassociated with POIs that are farther from the mobile device 120 can berendered as increasingly transparent based on their distance from thecurrent position of the mobile device 120.

When the mobile device 120 is configured to operate in the X-ray view,the rendering module 462 can be configured to provide an augmented viewof the indoor environment that can illustrate details of POIs that wouldotherwise be blocked by objects in the field of view of the mobiledevice 120, including components of the indoor environment and/or otherPOIs. When rendering the X-ray view, the rendering module 462 can beconfigured to first obtain a set of POIs from the POI module 464 asdescribed above. Each POI can include one or more images associated withthe POI, and the rendering module 462 can be configured to render acomposite image of the indoor environment based on the series of imagesand/or video captured by the mobile device 120 and the images associatedwith the POIs that fall within the field of view of the mobile device120. The rendering module 462 can be configured to request that the POIsretrieved by the POI module 464 be ordered based on the distance betweeneach POI and the current position of the mobile device 120. Therendering module 462 can then generate an augmented view where an imageassociated with the POI is rendered as semi-transparent based on thedistance of the POI from the current position of the mobile device 120.In one implementation, the rendering module 462 can be configured to usealpha compositing to generate the augmented view where the imageassociated with each POI is associated with an alpha channel value thatfalls within a range of completely opaque to completely transparent. Therendering module 462 can be configured to assign an alpha channel valuerepresenting a higher transparency to those POIs farther from thecurrent position of the mobile device and an alpha channel valuerepresenting a lower transparency to those POIs closer to the currentposition of the mobile device 120. The rendering module 462 can then beconfigured to render a composite image of the field of view of themobile device 120 by compositing the field of view content captured bythe camera 215 of the mobile device 215 with the POI images, startingwith those POIs farther from the current position of the mobile device120. The resulting composite image provides X-ray view of the indoorenvironment that can allow the user to see through walls of the indoorenvironment to see details that the user would not otherwise be able tosee from their vantage point. As described above, FIGS. 11A, 11B, 11C,and 11D illustrate an example of an X-ray view.

The rendering module 462 can be configured to request and receiveupdates to the position of the mobile device 120 from the mobile device120 and/or the location server 150. The rendering module 462 can beconfigured to automatically receive updates from the mobile device 120and/or the location server 150 when the position of the mobile device120 changes. The rendering module 462 can be configured to receiveupdates from the mobile device 120 when the orientation of the mobiledevice changes 120. The rendering module 120 can be configured toautomatically update the augmented view (directional view, X-ray view,or other augmented view) generated for display on the display 225 of themobile device in response to changes to the position and/or theorientation of the mobile device 120 and to send the updated augmentedview to the mobile device 120.

The point of interest (POI) module 464 can be configured to identifypoints of interest within the field of view of the mobile device 120based on position and orientation information for the mobile device 120.The position information can be received from the mobile device 120and/or the location server 150, and the orientation information can bereceived from the mobile device 120. The rendering module 462 can beconfigured to obtain the current position of the mobile device 120within the indoor environment from the mobile device 120 and/or from thelocation server 150 and to provide the location of the mobile device 120to the point of interest module 464. The rendering module 462 can alsobe configured to obtain the orientation information for the mobiledevice 120. The POI module 464 can be configured to operate similarly asthe POI module 464 of the mobile device 120. The POI module 464 can beconfigured to use the position and orientation information to identify aset of one or more POIs that are in the direction of the field of viewof the mobile device 120. As indicated above, the POIs may or may not bevisible from the current position of the mobile device 120. The POImodule 464 can be configured to order the set of POIs by their relativedistance from the current position of the mobile device 120 and can beconfigured to mark each POI to indicate whether the POI may be visiblefrom the current position of the mobile device 120 or whether the POI isobscured behind another component of the indoor environment.

The POI module 464 can be configured to access POI data from thenavigation database 450 via the data access module 466. The data accessmodule 466 can be configured to access the POI information for aparticular indoor environment or portion of an indoor environment, suchas a floor of the indoor environment, and to provide the POI data to thePOI module 464. The navigation database 450 can include similar data asthe navigation database 350 of the mobile device 120 as described above.The navigation database 450 can include POI information associated withPOIs, including geometry information defining the boundaries of the POIand/or coordinates of the POI in either a global or self-defined, localcoordinate system associated with an indoor environment. The POIinformation can also include images of the POIs that can be used torepresent the POI in the X-ray view of the indoor environment and caninclude textual and/or graphical labels that can be used to representthe POI in the directional view of the indoor environment provided bythe rendering engine 462.

The request processing module 468 can be configured to receive requestsfor information from a mobile device 120 and to send requested data tothe mobile device 120. For example, the request processing module 468can be configured to receive requests for augmented content fornavigating through an indoor environment. The request processing module468 can be configured to request information from the data access module466 and/or the rendering module 462 if the requested information is notavailable in the navigation database 450. The request processing module468 can also be configured to send the requested information to themobile device 120 once the requested information has been provided bythe data access module 466 and/or the rendering module 462.

The data access module 466 can be configured to access data stored inthe memory 460 of the mobile device 120, the user roles database 475,and/or in the navigation database 450. The data access module 466 can beconfigured to access data stored in or to store data in the memory 460and/or in the navigation database 450 in response to requests from therendering module 462, the point of interest module 464, and/or otherapplications or modules implemented on the navigation server 125.

FIG. 5 is a functional block diagram of a computer system that can beused to implement the location server 150 illustrated in FIG. 1. In theexamples provided in the figures, the navigation server 125 and thelocation server 150 are illustrated as separate servers. However, inalternative implementations, some or all of the functionality of thenavigation server 125 and the location server 150 may be implemented onthe same server.

The location server 150 can include a processor 555, a network interface550, and a memory 560. The location server 150 can also includefunctional modules that can be implemented as processor executablesoftware code stored in memory 560. The functional modules illustratedin FIG. 5 can alternatively be stored either in part or in whole in adifferent computer-readable memory. For example, the functional modulesmay be stored in a removable computer-readable medium associated withthe location server 150, such as a flash memory, a SD-Card, or othertype of removable memory. The functional modules illustrated in FIG. 5can also be implemented in part or in whole in firmware and/or hardware.The navigation server 125 can include a position determination module562 and an assistance data module 564.

The processor 555 can be configured to execute software code stored inthe memory 560. For example, the processor 555 can be configured toexecute the software code associated with the position determinationmodule 562 and the assistance data module 564. The processor 555 canalso be configured to execute other software code used by the locationserver 150, such as operating system components, software drivers forperipheral devices, and/or other software code that may be executed bythe location server 150 but the description of which is not required tounderstand the operation of the systems and methods disclosed herein.

The network interface 550 can be configured to provide bidirectionalwireless and/or wired network communications to the location server 150.The network interface 550 can be configured to allow the location server150 to communicate with mobile devices 120 via one or more intermediatenetworks, such as the Internet, a wireless network service provider'score network, one or more wireless local area networks (WLANs), and/orother types of network. The network communications between the networkinterface 550 of the location server 150 and the mobile devices 120 maybe routed through one or more other network elements, such as wirelessaccess point 115 or base station 140.

The memory 560 can comprise volatile and/or persistent memory forstoring data used by various components of the location server 150. Thememory 460 can be used to store processor-executable program code forone or more of the processors included in the location server 150, suchas processor 555.

The position determination module 562 can be configured to providelocation based services to mobile devices 120. For example, the positiondetermination module 562 can be configured to determine the position ofa mobile device 120 and/or provide assistance data to the mobile device120 that can be used to determine the position of the mobile device 120within an indoor environment and/or in an outdoor environment. Fordetermining the position of a mobile device 120 within an indoorenvironment, the position determination module 562 can be configured toobtain from the mobile device 120 measurements of signals from aplurality of wireless access points 115 to estimate the location of themobile device 120 within an indoor environment in various techniques,such as RSSI, RTT, TOA, and/or other techniques. The positiondetermination module 562 can also be configured to determine theposition of the mobile device 120 using signals received from aSatellite Positioning System receiver associated with the mobile device120 (e.g., associated with communications controller 250) to determinethe position of the mobile device 120 when outdoors and a line of siteto a sufficient number of satellites from one or more SPSconstellations.

The assistance data module 564 can be configured to identify assistancedata that the mobile device 120 can use to determine the location of themobile device 120. The assistance data module 564 can be configured toprovide the mobile device 120 with an almanac of wireless access points115 and/or base stations 140 which may be proximate to the mobile device120. The mobile device 120 can use this information to identify wirelessaccess points 115 and/or base stations 140 that may be proximate to themobile device 120 and to use signals from nearby wireless access points115 and/or base stations 140 to determine the position of the mobiledevice 120.

FIG. 6 is a block flow diagram of a process for displaying navigationinformation on a mobile device 120. The process illustrated in FIG. 6can be implemented by the mobile device 120. However, some or all of thestages of the process illustrated in FIG. 6 can be implemented either inwhole or in part by the navigation server 125 and/or the location server150.

The process can begin with obtaining an indication of a position of themobile device 120 within a geographical area of interest, such as anindoor environment (stage 605). The position of the mobile device 120can be determined by the mobile device 120 and/or by the location server150. For example, the position determination module 370 of the mobiledevice 120 can be configured to determine the location of the mobiledevice 120. The position determination module 370 can be configured touse signals from wireless access points 115 and/or base stations 140 todetermine the location of the mobile device 120. The positiondetermination module 370 can also be configured to use assistance datareceived from the location server 150 to determine the position of themobile device 120. The position of the mobile device 120 can also bedetermined by the location server 150 and the location server 150 can beconfigured to send the position information to the mobile device 120.

An indication of the direction associated with the mobile device 120 canalso be obtained (stage 610). The mobile device 120 can be configured todetermine the orientation of the mobile device 120 based on sensor datacollected by the orientation determination module 366. The orientationof the mobile device 120 can be used to determine how the mobile device120 is oriented relative to the floor, walls, ceiling, and/or otherelements of an indoor environment. Based on the orientation of themobile device 120, it is possible to determine which POIs should fallwithin the field of view of the camera 225. The POIs may not necessarilybe visible and may be behind walls, floors, ceilings, and/or othercomponents of an indoor environment.

At least one relevant point of interest within the geographical regionof interest can be determined based on the position of the mobile device120, the direction associated with the mobile device 120, and POIinformation associated with the geographic area of interest (stage 615).Multiple POIs can be associated with a geographical area of interest.For example, an indoor environment, such as a shopping mall, can includePOIs identifying the location of stores and restaurants and/orcomponents of stores, such as cashier locations, the women's clothingsection, menswear section, restroom facilities, changing rooms, and/orother aspects of a store that may be of interest of someone shopping atthat location. In another example, POIs for a government building mightinclude the location of the Department of Motor Vehicles, the CountyRecorder's Office, and the Department of Health and Human Safety. In yetanother example, the POIs for an office building might include thelocation of a reception area or lobby, the location of restrooms,conference rooms, and/or specific companies if the building housesmultiple tenants.

The at least one relevant point of interest can be determined usingvarious techniques, including, but not limited to the processillustrated in FIG. 7, which is described in detail below. The at leastone relevant point of interest can be determined by plotting a viewingdirection or field of view of the camera 215 of the mobile device 120.The camera 215 of the mobile device 120 can be configured capture videoand/or a series of images of the area within the field of view of thecamera 215 and to display the captured content on the display 225 of themobile device 120. The mobile device 120 can be configured to determinewhich POIs are within the field of view of the camera based on theposition of the camera with in the geographic region of interest, e.g.an indoor environment and the direction that the mobile device 120 isfacing. A set of one or more POIs that fall within the field of view ofthe camera 215 of the mobile device 120 can be selected from a set ofPOIs associated with a geographic area of interests, such as an indoorenvironment. The point of interest module 364 of the mobile device 120can be configured to identify the set of one or more relevant POIsand/or the mobile device 120 can be configured to receive the POIinformation from the navigation server 125. The request processingmodule 468 of the navigation server 125 can be configured to receive arequest for POI information from the mobile device 120. The mobiledevice 120 can be configured to provide the location and orientationinformation associated with the mobile device 120 to the navigationserver 125 with the request for the POI information. The POI informationcan include geometry information defining the boundaries of the POIand/or coordinates of the POI in either a global or self-defined, localcoordinate system associated with an indoor environment. The POIinformation can also include images of the POIs that can be used torepresent the POI in the X-ray view of the indoor environment and caninclude textual and/or graphical labels that can be used to representthe POI.

A visual indication can then be displayed on the mobile device 120 forthe at least one relevant point of interest (stage 620). The mobiledevice 120 can be configured to display a directional view, an X-rayview, or a combination thereof on the display of the mobile device 120.As described above the directional view can be provide an augmented viewof an indoor environment that includes overlays of textual and/orgraphical indications illustrating the position of POIs within theindoor environment relative to the position of the mobile device 120.The textual and/or graphical indications can be scaled in size relativeto their distance from the position of the mobile device 120. Textualand/or graphical indications can also be displayed for POIs that fallwithin the field of view of the camera 215 but are not visible in theimages and/or video content captured by the camera 215 because the POIsare blocked from view by one or more components of the indoorenvironment. The X-ray view can provide an augmented view of the indoorenvironment that can illustrate details of POIs that would otherwise beblocked by objects in the field of view of the mobile device 120,including components of the indoor environment and/or other POIs. TheX-ray view can provide a view of images and/or video content captured bythe camera 215 of the mobile device 120 composited with images of thePOIs that would otherwise be blocked from view by components of theindoor environment, such as floors, doors, walls, ceilings, and/or othercomponents of the indoor environment. The composite image can be formedby ordering the POIs based on their relative distances from the positionof the mobile device 120 and assigning an alpha channel value to imagesassociated with each POI based on the relative distance of the POI fromthe position of the mobile device 120. A composite image of the field ofview of the mobile device 120 can then be rendered by compositing thefield of view content captured by the camera 225 with the POI images,starting with those POIs farther from the current position of the mobiledevice 120.

FIG. 7 is a block flow diagram of a process for determining a relevantset of POIs. Aspects of the process illustrated in FIG. 7 can beimplemented by the mobile device 120 and/or by the navigation server125. The process illustrated in FIG. 7 can be used to implement, atleast in part, stage 615 of the process illustrated in FIG. 6.

An eye ray associated with the position and the indication of directionof the mobile device 120 can be determined (stage 705). The eye ray cancomprise imaginary line extending from the position of the mobile device120 in the direction associated with the indication of directionobtained for the mobile device 120 in stage 610 of the processillustrated in FIG. 6. The eye ray can be used to determine which POIscould fall within the field of view of the camera 215 of the mobiledevice 120. The eye ray can also be used to determine which POIs couldfall within the field of view of the camera 215 of the mobile device 120if the camera 215 were able to peer through physical barriers in thegeographic area of interest, such as floors, walls, ceilings, and otherobjects to view POIs that would otherwise be obscured.

In some implementations, the eye ray can be projected from a positioncentered at the position of the mobile device 120. In someimplementations, the height of mobile device 120 above the floor orground plane of the geographic area of interest can be taken intoaccount when determining the eye ray. For example, the sensors 205 ofthe mobile device 120 may include one or more sensors that are capableof determining the altitude of the mobile device 120. The mobile device120 and/or the navigation server 125 can also be configured to determinean altitude of the mobile device 120 based on RSSI, RTT, and/ormeasurement of signals transmitted between the mobile device 120 and oneor more wireless access points 115, base stations 140, or other mobiledevices 120 at a known position. In some implementations, a defaultaverage value can be used for the height from the floor based on anaverage height for the typical user of a mobile device 120 and/or atypical height at which an average user would hold the mobile device 120above the floor or ground plane of a geographic area of interest whenusing the mobile device 120 to navigate through the geographic area ofinterest.

In some implementations, multiple eye rays can be determined andprojected through the geometric representation of the geographic area ofinterest. Sometimes a single eye ray may not be sufficient to identifyall of the POIs that fall within the field of view. Accordingly,multiple eye rays may be determined to provide more complete informationabout which POIs that fall within the field of view of the mobile device120 (and consequently a user of the mobile device 120). In someimplementations, multiple eye rays can be determined by uniformlysampling the angle of the field of view, and an intersection testsimilar to that used for the single eye ray example can be used todetermine whether any of the eye rays intersection with a POI. If one ormore eye rays intersect with a POI, that POI can be displayed in theaugmented view of the region of geographic interest presented on thedisplay of the mobile device 120.

The eye ray can then be projected through a geometric representation ofthe geographic area of interest (stage 710). As described above, each ofthe POIs can be associated with POI information that can includegeometry information defining the boundaries of the POI and/orcoordinates of the POI in either a global or self-defined, localcoordinate system associated with an indoor environment. The geometryinformation can provide information defining 2D boundaries for aparticular POI or can provide 3D boundaries of associated with the POI.In some implementations, an area of geographic interest can comprise amultistory building or buildings and the eye-ray can be projected acrossmultiple floors of a multistory environment to determine whether theeye-ray intersects with one or more POIs on multiple floors of themultistory environment.

At least one POI with which the eye ray intersects can be identified(stage 715). Based on the projection of the eye-ray through thegeographic area of interest, POIs with which the eye ray intersectionscan be selected as relevant points of interest. The eye ray can be usedto represent what the field of view of the camera might include if thecamera 215 could view peer through physical barriers and/or objects inthe geographical area of interest. The POIs identified using the processillustrated in FIG. 7 can be displayed on the display 225 of the mobiledevice 120 to provide a user of the mobile device 120 with contextualinformation that may be useful for navigating through the geographicarea of interest.

The POI information can include geometry information defining theboundaries of the POI and/or coordinates of the POI in either a globalor self-defined, local coordinate system associated with an indoorenvironment. The POI information can be used to determine how far eachof the POIs are from the current position of the mobile device 120 andthe POIs can be sorted based on that distance from the current positionof the mobile device 120.

FIG. 8 is a block flow diagram of a process for determining displayingon the display 225 of the mobile device 120 a directional view ofrelevant POIs within a field of view of a camera 215 of a mobile device120. The process illustrated in FIG. 8 can be implemented by the mobiledevice 120 and/or by the navigation server 125. The process illustratedin FIG. 7 can be used to implement, at least in part, stage 620 of theprocess illustrated in FIG. 6. FIG. 10, described above, provides anexample of a mobile device 120 displaying directional view of ageographical area of interest that comprises an indoor shopping mall.

The POIs identified based on the position and direction of the mobiledevice 120 can be sorted based on their distance from the mobile device120 (stage 805). Each of the POIs can be associated with POI informationThe POI information can include geometry information defining theboundaries of the POI and/or coordinates of the POI in either a globalor self-defined, local coordinate system associated with an indoorenvironment. The POI information can be used to determine how far eachof the POIs are from the current position of the mobile device 120 andthe POIs can be sorted based on that distance from the current positionof the mobile device 120.

Textual and/or graphical indications associated with the POIs can thenbe rendered (stage 810). The POI information associated with each of thePOIs can include textual and/or graphical indications that can berendered to represent that POI. For example, the POI information cancontain textual information, such as a name or description of the POI.In one example implementation, the directional view can be provided foran indoor shopping mall, and the POIs can be associated with textualindications, such as a store or restaurant name and/or a description ofthe POI (e.g., fitting rooms, restrooms, etc.). The POI information canalso include graphical indications, such as a store or restaurant logoor an icon representing a type of retailer (e.g. clothing, departmentstore, furniture, shoes, appliances, etc.). In another exampleimplementation, the POI information can be associated with an officebuilding where the textual and/or graphical indications can becustomized for an office environment. In such an implementation, the POIinformation can include textual and/or graphical indications identifyingspecific business and/or portions of a business, such as copy rooms,break rooms, conference rooms, supply rooms, etc.

The textual and/or graphical indications can be rendered as an overlayover images and/or video content captured by the camera 215 of themobile device 120 to create an augmented view of the geographic area ofinterest within the field of view of the camera. In someimplementations, the size of the textual and/or graphical indicationsrendered in the augmented view can be based how close the POI associatedwith the textual and/or graphical indications is to the current positionof the mobile device 120. The textual and/or graphical indications canbe rendered larger if the POI is closer to the current position of themobile device 120 and the textual and/or graphical indications can berendered smaller if the POI is farther from the position of the mobiledevice 120.

In some implementations, the POI may be obscured in the field of view inthe images and/or video content by one or more obstacles between thecurrent position of the camera and the position of the POI, but textualand/or graphical indications can be displayed for such POIs to providethe user of the mobile device 120 with contextual information about thecurrent position of the mobile device 120. For example, a wall, floor,ceiling, or other component of the building may obscure the POI from thecamera 215 at the current position of the mobile device 120. In anotherexample, one or more objects within the region of interest, such aspeople, fixtures within a store, etc. that are not structural componentsof a building could also obscure the POI from the field of view of thecamera 215 of the mobile device, but textual and/or graphicalindications could still be provided to guide a user of the mobile device120 toward the obscured POIs. In some implementations, the renderingmodule 362 of the mobile device and/or the rendering module 462 of thenavigation server 125 can be configured to render the textual and/orgraphical indications associated with POIs that are obstructed from viewby one or more components of a building or other indoor environment aspartially transparent to indicate to a user of the mobile device 120that the POI is obscured by components of the building or other indoorenvironment. The POIs that are obstructed from view by components of theindoor environment can be identified based on the current position ofthe mobile device 120, the direction that the mobile device 120 isfacing and/or the orientation of the device, 2D or 3D map informationfor an indoor environment, and POI information defining the geometry ofthe POIs within the indoor environment.

In some implementations, the sorting of stage 805 can be optional andthe textual and/or graphical indications associated with the POIs canthen be rendered based on their relative distance from the currentposition of the mobile device 120 without first sorting the POIs.

The rendered indications associated with the POIs can then be displayedwith representation of geographic area of interest (stage 815). Therendered textual and/or graphical indications associated with the POIscan be displayed on the display 225 of the mobile device 120. In someimplementations, the rendered textual and/or graphical indications canbe displayed as an overlay to the image and/or video content captured ofthe field of view of the camera 215. In other implementations, therendering module 362 of the mobile device 120 and/or the renderingmodule 462 of the navigation server can be configured to generated imageand/or video content for display on the display 225 that incorporatesthe textual and/or graphical indications associated with the POIs withthe image and/or video content of the field of view of the camera 215 ofthe mobile device 120. The latter approach can be used where therendered textual and/or graphical indications are associated with alevel of transparency and are not rendered completely as a complexlyopaque overlay to the content captured by the camera 215.

FIG. 9 is a block flow diagram of a process for determining displayingan X-ray view of POIs within a field of view of a camera 215 of a mobiledevice 120 on a display 225 of the mobile device 120. The processillustrated in FIG. 8 can be implemented by the mobile device 120 and/orby the navigation server 125. The process illustrated in FIG. 7 can beused to implement, at least in part, stage 620 of the processillustrated in FIG. 6.

The POIs identified based on the position and direction of the mobiledevice 120 can be sorted based on their distance from the mobile device120 (stage 905). Each of the POIs can be associated with POI informationThe POI information can include geometry information defining theboundaries of the POI and/or coordinates of the POI in either a globalor self-defined, local coordinate system associated with an indoorenvironment. The POI information can be used to determine how far eachof the POIs are from the current position of the mobile device 120 andthe POIs can be sorted based on that distance from the current positionof the mobile device 120.

An alpha channel value can be determined for each of the POIs based onthe distance of the POI from the position of the mobile device 120(stage 910). In some implementations, the sorting of stage 905 can beoptional and an alpha channel value can be determined for each POI basedon its relative distance from the current position of the mobile device120. The alpha channel value associated with each of the POIs canindicate how transparent (or conversely how opaque) indicationsassociated with a particular POI should be rendered. In someimplementations, the alpha channel values can be determined such thatindications associated with POIs that are within a predetermineddistance from the position of the mobile device 120 are rendered asopaque, while the indications associated with those POIs that arefarther than the predetermined distance from the position of the mobiledevice 120 are rendered as increasingly transparent the farther the POIis from the position of the mobile device 120. A threshold distance canalso be defined beyond which indications associated with POIs at orbeyond that distance are not rendered at all (e.g., rendered at 100%transparency).

Composite image content can be generated based on the image associatedwith POI, the sort order of the POIs, and the alpha channel valuesassociated with each of the POIs (stage 915). For example, each of thePOIs can be associated with an image of the POI or other graphicalrepresentation of the POI. The composite image content can be generatedby combining the textual and/or graphical indicia associated with eachof the POIs with image and/or video content of the field of the view ofthe mobile device 120 captured by the camera 215. In one exampleimplementation, the image and/or video content of the field of view ofthe mobile device 120 captured by the camera can be associated with apredetermined transparency level and the textual and/or graphicalindications associated with each of the POIs can then be rendered overthe image and/or video content of the field of view of the mobile device120 captured by the camera. In some implementations, the textual and/orgraphical indications associated with the POIs can be rendered based onthe distance of each of the POIs from the position of the mobile device120, starting with the POIs that is farthest from the position of themobile device 120 first and ending with the POI that is closes to theposition of the mobile device 120.

The composite image content can then be displayed on the display 225 ofthe mobile device 120 (stage 920). The composite augmented view of thefield of view of the camera that includes the indications of the POIrendered in stage 915 can be displayed on the display 225 of the mobiledevice 120 to provide an X-ray view of a geographical area of interest.

Other Considerations

As used herein, including in the claims, “or” as used in a list of itemsprefaced by “at least one of” indicates a disjunctive list such that,for example, a list of “at least one of A, B, or C” means A or B or C orAB or AC or BC or ABC (i.e., A and B and C), or combinations with morethan one feature (e.g., AA, AAB, ABBC, etc.).

As used herein, including in the claims, unless otherwise stated, astatement that a function or operation is “based on” an item orcondition means that the function or operation is based on the stateditem or condition and may be based on one or more items and/orconditions in addition to the stated item or condition.

A wireless communication network does not have all communicationstransmitted wirelessly, but is configured to have at least somecommunications transmitted wirelessly.

Other examples and implementations are within the scope and spirit ofthe disclosure and appended claims. For example, due to the nature ofsoftware, functions described above can be implemented using softwareexecuted by a processor, hardware, firmware, hardwiring, or combinationsof any of these. Features implementing functions may also be physicallylocated at various positions, including being distributed such thatportions of functions are implemented at different physical locations.Also, as used herein, including in the claims, “or” as used in a list ofitems prefaced by “at least one of” indicates a disjunctive list suchthat, for example, a list of “at least one of A, B, or C” means A or Bor C or AB or AC or BC or ABC (i.e., A and B and C), or combinationswith more than one feature (e.g., AA, AAB, ABBC, etc.).

Further, more than one invention may be disclosed.

Substantial variations to described configurations may be made inaccordance with specific requirements. For example, customized hardwaremight also be used, and/or particular elements might be implemented inhardware, software (including portable software, such as applets, etc.),or both. Further, connection to other computing devices such as networkinput/output devices may be employed.

Common forms of physical and/or tangible computer-readable mediainclude, for example, a floppy disk, a flexible disk, hard disk,magnetic tape, or any other magnetic medium, a CD-ROM, any other opticalmedium, punchcards, papertape, any other physical medium with patternsof holes, a RAM, a PROM, EPROM, a FLASH-EPROM, any other memory chip orcartridge, a carrier wave as described hereinafter, or any other mediumfrom which a computer can read instructions and/or code.

The methods, systems, and devices discussed above are examples. Variousconfigurations may omit, substitute, or add various procedures orcomponents as appropriate. For instance, in alternative configurations,the methods may be performed in an order different from that described,and that various steps may be added, omitted, or combined. Also,features described with respect to certain configurations may becombined in various other configurations. Different aspects and elementsof the configurations may be combined in a similar manner. Also,technology evolves and, thus, many of the elements are examples and donot limit the scope of the disclosure or claims.

Specific details are given in the description to provide a thoroughunderstanding of example configurations (including implementations).However, configurations may be practiced without these specific details.For example, well-known circuits, processes, algorithms, structures, andtechniques have been shown without unnecessary detail in order to avoidobscuring the configurations. This description provides exampleconfigurations only, and does not limit the scope, applicability, orconfigurations of the claims. Rather, the preceding description of theconfigurations provides a description for implementing describedtechniques. Various changes may be made in the function and arrangementof elements without departing from the spirit or scope of thedisclosure.

Also, configurations may be described as a process which is depicted asa flow diagram or block diagram. Although each may describe theoperations as a sequential process, many of the operations can beperformed in parallel or concurrently. In addition, the order of theoperations may be rearranged. A process may have additional stages orfunctions not included in the figure. Furthermore, examples of themethods may be implemented by hardware, software, firmware, middleware,microcode, hardware description languages, or any combination thereof.When implemented in software, firmware, middleware, or microcode, theprogram code or code segments to perform the tasks may be stored in anon-transitory computer-readable medium such as a storage medium.Processors may perform the described tasks.

Having described several example configurations, various modifications,alternative constructions, and equivalents may be used without departingfrom the spirit of the disclosure. For example, the above elements maybe components of a larger system, wherein other rules may takeprecedence over or otherwise modify the application of the invention.Also, a number of operations may be undertaken before, during, or afterthe above elements are considered. Accordingly, the above descriptiondoes not bound the scope of the claims.

What is claimed is:
 1. A method of generating one or more augmentedviews, the method comprising: obtaining an image of a scene captured bya camera disposed within a geographic area of interest including a firstpoint of interest and a second point of interest, the first point ofinterest being disposed at a first distance relative to the camera andthe second point of interest being disposed at a second distancerelative to the camera, the first distance being different than thesecond distance; and generating an augmented view, wherein generatingthe augmented view includes: overlaying over the scene, at a firsttransparency level, a first visual indication associated with the firstpoint of interest; and overlaying over the scene, at a secondtransparency level, a second visual indication associated with thesecond point of interest, the first visual indication overlapping atleast a portion of the second visual indication, wherein the secondtransparency level of the second visual indication is different than thefirst transparency level of the first visual indication.
 2. The methodof claim 1, wherein the first visual indication overlaps at least aportion of the second visual indication based on the first point ofinterest being disposed at least partially behind the second point ofinterest in the scene relative to the camera.
 3. The method of claim 1,further comprising displaying the augmented view on a mobile device. 4.The method of claim 1, wherein at least a portion of the first visualindication does not overlap at least a portion of the second visualindication.
 5. The method of claim 1, wherein at least one of the firstvisual indication or the second visual indication has a size that isdependent on a distance from a device including the camera to at leastone of the first point of interest or the second point of interest. 6.The method of claim 1, wherein the second transparency level has a lowertransparency than the first transparency level.
 7. The method of claim1, wherein one or more visual indications associated with one or morepoints of interest that are not visible in a field of view of the cameraare displayed as at least partially transparent based on the one or morepoints of interest being at least partially obstructed from view by oneor more objects in the geographic area of interest.
 8. The method ofclaim 1, wherein a transparency level of each of one or more visualindications associated with one or more points of interest is dependenton a presence of one or more physical barriers between a deviceincluding the camera and the one or more points of interest.
 9. Themethod of claim 1, wherein one or more visual indications associatedwith one or more points of interest are displayed as opaque based on theone or more visual indications being associated with one or more pointsof interest that are visible in a field of view of the camera.
 10. Themethod of claim 1, further comprising generating an updated augmentedview, wherein at least one of the first transparency level of the firstvisual indication or the second transparency level of the second visualindication is adjusted in the updated augmented view based on movementof a device relative to at least one of the first point of interest orthe second point of interest.
 11. The method of claim 1, wherein thegeographic area of interest comprises an indoor environment.
 12. Themethod of claim 11, wherein a device including the camera is in theindoor environment, wherein the second point of interest is in theindoor environment, and wherein the first point of interest is outsideof the indoor environment.
 13. The method of claim 1, wherein a deviceincluding the camera is in a first indoor environment, wherein thesecond point of interest is in the first indoor environment, and whereinthe first point of interest is in a second indoor environment.
 14. Themethod of claim 1, wherein the geographic area of interest comprises anoutdoor environment.
 15. The method of claim 1, wherein a deviceincluding the camera is in a first outdoor environment, wherein at leastone of the second point of interest and the first point of interest isin a second outdoor environment.
 16. The method of claim 1, wherein thefirst transparency level of the first visual indication and the secondtransparency level of the second visual indication are based on thefirst point of interest being disposed at least partially behind thesecond point of interest in the geographic area of interest.
 17. Themethod of claim 1, wherein the first transparency level of the firstvisual indication is based on the first distance and the secondtransparency level of the second visual indication is based on thesecond distance.
 18. The method of claim 1, wherein the first point ofinterest is disposed at a distance from the second point of interest,and wherein the first transparency level of the first visual indicationand the second transparency level of the second visual indication arebased on the distance.
 19. The method of claim 1, wherein the firstvisual indication is displayed as being in front of at least a portionof the second visual indication based on the first distance beinggreater than the second distance.
 20. The method of claim 1, wherein thefirst visual indication includes a first image, and wherein the secondvisual indication includes a second image.
 21. The method of claim 1,wherein the first visual indication is overlaid over the image of thescene at the first transparency level, and wherein the second visualindication is overlaid over the image of the scene at the secondtransparency level.
 22. An apparatus for generating one or moreaugmented views, comprising: a memory configured to store one or moreimages; and a processor configured to: obtain an image of a scenecaptured by a camera disposed within a geographic area of interestincluding a first point of interest and a second point of interest, thefirst point of interest being disposed at a first distance relative tothe camera and the second point of interest being disposed at a seconddistance relative to the camera, the first distance being different thanthe second distance; and generate an augmented view, wherein generatingthe augmented view includes: overlaying over the scene, at a firsttransparency level, a first visual indication associated with the firstpoint of interest; and overlaying over the scene, at a secondtransparency level, a second visual indication associated with thesecond point of interest, the first visual indication overlapping atleast a portion of the second visual indication, wherein the secondtransparency level of the second visual indication being is differentthan the first transparency level of the first visual indication. 23.The apparatus of claim 22, wherein the first visual indication overlapsat least a portion of the second visual indication based on the firstpoint of interest being disposed at least partially behind the secondpoint of interest in the scene relative to the camera.
 24. The apparatusof claim 22, further comprising a display, wherein the processor isfurther configured to cause the augmented view to be displayed on thedisplay.
 25. The apparatus of claim 22, wherein at least a portion ofthe first visual indication does not overlap at least a portion of thesecond visual indication.
 26. The apparatus of claim 22, wherein thesecond transparency level has a lower transparency than the firsttransparency level.
 27. The apparatus of claim 22, wherein one or morevisual indications associated with one or more points of interest thatare not visible in a field of view of the camera are displayed as atleast partially transparent based on the one or more points of interestbeing at least partially obstructed from view by one or more objects inthe geographic area of interest.
 28. The apparatus of claim 22, whereina transparency level of each of one or more visual indicationsassociated with one or more points of interest is dependent on apresence of one or more physical barriers between a device including thecamera and the one or more points of interest.
 29. The apparatus ofclaim 22, wherein one or more visual indications associated with one ormore points of interest are displayed as opaque based on the one or morevisual indications being associated with one or more points of interestthat are visible in a field of view of the camera.
 30. The apparatus ofclaim 22, wherein the processor is further configured to generate anupdated augmented view, wherein at least one of the first transparencylevel of the first visual indication or the second transparency level ofthe second visual indication is adjusted in the updated augmented viewbased on movement of a device relative to at least one of the firstpoint of interest or the second point of interest.
 31. The apparatus ofclaim 22, wherein the first transparency level of the first visualindication is based on the first distance and the second transparencylevel of the second visual indication is based on the second distance.32. The apparatus of claim 22, wherein the first point of interest isdisposed at a distance from the second point of interest, and whereinthe first transparency level of the first visual indication and thesecond transparency level of the second visual indication are based onthe distance.
 33. The apparatus of claim 22, wherein the apparatusincludes the camera.
 34. The apparatus of claim 22, wherein thegeographic area of interest comprises an indoor environment, wherein theapparatus includes the camera and is in the indoor environment, whereinthe second point of interest is in the indoor environment, and whereinthe first point of interest is outside of the indoor environment. 35.The apparatus of claim 22, wherein the geographic area of interestcomprises a first indoor environment, wherein the apparatus includes thecamera and is in the first indoor environment, wherein the second pointof interest is in the first indoor environment, and wherein the firstpoint of interest is in a second indoor environment.
 36. The apparatusof claim 22, wherein the geographic area of interest comprises a firstoutdoor environment, wherein the apparatus includes the camera and is inthe first outdoor environment, and wherein at least one of the secondpoint of interest and the first point of interest is in a second outdoorenvironment.
 37. The apparatus of claim 22, wherein the first visualindication is overlaid over the image of the scene at the firsttransparency level, and wherein the second visual indication is overlaidover the image of the scene at the second transparency level.
 38. Theapparatus of claim 22, further comprising: a transmitter fortransmitting the augmented view to a device including the camera. 39.The apparatus of claim 22, wherein the first visual indication isdisplayed as being in front of at least a portion of the second visualindication based on the first distance being greater than the seconddistance.
 40. A non-transitory computer-readable medium having storedthereon instructions that, when executed by one or more processors,cause the one or more processors to: obtain an image of a scene capturedby a camera disposed within a geographic area of interest including afirst point of interest and a second point of interest, the first pointof interest being disposed at a first distance relative to the cameraand the second point of interest being disposed at a second distancerelative to the camera, the first distance being different than thesecond distance; and generate an augmented view, wherein generating theaugmented view includes: overlaying over the scene, at a firsttransparency level, a first visual indication associated with the firstpoint of interest; and overlaying over the scene, at a secondtransparency level, a second visual indication associated with thesecond point of interest, the first visual indication overlapping atleast a portion of the second visual indication, wherein the secondtransparency level of the second visual indication is different than thefirst transparency level of the first visual indication.
 41. Thenon-transitory computer-readable medium of claim 40, wherein the firstvisual indication is displayed as being in front of at least a portionof the second visual indication based on the first distance beinggreater than the second distance.